A Characterization of Entropy in Terms of Information Loss

نویسندگان

  • John C. Baez
  • Tobias Fritz
  • Tom Leinster
چکیده

There are numerous characterizations of Shannon entropy and Tsallis entropy as measures of information obeying certain properties. Using work by Faddeev and Furuichi, we derive a very simple characterization. Instead of focusing on the entropy of a probability measure on a finite set, this characterization focuses on the “information loss”, or change in entropy, associated with a measure-preserving function. Information loss is a special case of conditional entropy: namely, it is the entropy of a random variable conditioned on some function of that variable. We show that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous. This characterization naturally generalizes to Tsallis entropy as well.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Quantile Approach of Generalized Cumulative Residual Information Measure of Order $(alpha,beta)$

In this paper, we introduce the concept of quantile-based generalized cumulative residual entropy of order $(alpha,beta)$ for residual and past lifetimes and study their properties. Further we study the proposed information measure for series and parallel system when random variable are untruncated or truncated in nature and some characterization results are presented. At the end, we study gene...

متن کامل

E-Bayesian Approach in A Shrinkage Estimation of Parameter of Inverse Rayleigh Distribution under General Entropy Loss Function

‎Whenever approximate and initial information about the unknown parameter of a distribution is available, the shrinkage estimation method can be used to estimate it. In this paper, first the $ E $-Bayesian estimation of the parameter of inverse Rayleigh distribution under the general entropy loss function is obtained. Then, the shrinkage estimate of the inverse Rayleigh distribution parameter i...

متن کامل

Some Results Based on Entropy Properties of Progressive Type-II Censored Data

In many life-testing and reliability studies, the experimenter might not always obtain complete information on failure times for all experimental units. One of the most common censoring schemes is progressive type-II censoring. The aim of this paper is characterizing the parent distributions based on Shannon entropy of progressive type-II censored order statistics. It is shown that the equality...

متن کامل

Qualitative Assessment of Hydrometric Station Density Kermanshah Province Using the Discrete Entropy

Monitoring the quality parameters of the river is very important in the management of surface water resources. One of the effective methods in this field is entropy theory which has a very good capability in designing quality monitoring network. In this research, discrete entropy theory has been studied and monitored in order to qualitatively evaluate the temporal and spatial frequencies of the...

متن کامل

Some Results on Weighted Cumulative Entropy

Considering Rao et al. (2004) and Di Crescenzo and Longobardi (2009) studies, Misagh et al. (2011) proposed a weighted information which is based on the cumulative entropy called Weighted Cumulative Entropy (WCE). The above-mentioned model is a Shiftdependent Uncertainty Measure. In this paper, we examine some of the properties of WCE and obtain some bounds for that. In order to ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • Entropy

دوره 13  شماره 

صفحات  -

تاریخ انتشار 2011